Implement Code Indexing Feature for WSO2 Integrator Copilot#633
Implement Code Indexing Feature for WSO2 Integrator Copilot#633yasithrashan wants to merge 3 commits into
Conversation
📝 WalkthroughCode Indexing Feature Implementation for WSO2 Integrator CopilotCore Features AddedCodeMap Retrieval System
Code Exploration Tools
AI Agent Enhancements
Evaluation InfrastructureContext Retrieval Evaluation
Test Pipeline Updates
Test Data AdditionsAdded comprehensive sample projects for integration testing:
Configuration & Dependencies
Code Quality Improvements
WalkthroughThis pull request introduces a code map retrieval API for language server communication, implements grep and glob search tools powered by ripgrep, refactors the file read API to use line-based ranges, and adds comprehensive LLM-based context retrieval evaluation. The changes span core agent functionality, tool infrastructure, and multiple sample applications for testing. The code map feature establishes a new RPC interface and integrates it into the agent executor pipeline to provide high-level codebase summaries. The search tools leverage ripgrep with proper path validation and binary discovery. The text editor API migration from offset/limit to startLine/endLine improves clarity for line-based operations. Context retrieval evaluation introduces structured assessment of whether retrieved context is relevant to user queries. Test data includes complete implementations of healthcare, hotel reservation, order management, and integration scenarios across five distinct applications. ✨ Finishing Touches🧪 Generate unit tests (beta)
|
There was a problem hiding this comment.
Actionable comments posted: 18
🧹 Nitpick comments (2)
packages/ballerina-extension/test/ai/evals/code/result-management/report-generator.ts (2)
192-195: 💤 Low valueRemove redundant boolean display.
Same redundancy as in successful compilations section—the emoji already indicates the boolean state.
♻️ Proposed simplification
if (result.contextRetrievalEvaluation) { const ctx = result.contextRetrievalEvaluation; - console.log(` Context Retrieval: ${ctx.is_relevant ? '✅' : '❌'} Relevant: ${ctx.is_relevant}`); + console.log(` Context Retrieval: ${ctx.is_relevant ? '✅ Relevant' : '❌ Not Relevant'}`); }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/ballerina-extension/test/ai/evals/code/result-management/report-generator.ts` around lines 192 - 195, The console.log in report-generator.ts is printing both an emoji and the boolean for context relevance (using ctx.is_relevant) which is redundant; update the logging inside the block that checks result.contextRetrievalEvaluation so it only prints the emoji/state (e.g., keep `Context Retrieval: ${ctx.is_relevant ? '✅' : '❌'}`) and remove the trailing "Relevant: ${ctx.is_relevant}" portion, modifying the statement that references ctx and ctx.is_relevant accordingly.
166-169: 💤 Low valueRemove redundant boolean display.
The line shows both an emoji (✅/❌) and the boolean value text (
Relevant: ${ctx.is_relevant}). The emoji already conveys the boolean state.♻️ Proposed simplification
if (result.contextRetrievalEvaluation) { const ctx = result.contextRetrievalEvaluation; - console.log(` Context Retrieval: ${ctx.is_relevant ? '✅' : '❌'} Relevant: ${ctx.is_relevant}`); + console.log(` Context Retrieval: ${ctx.is_relevant ? '✅ Relevant' : '❌ Not Relevant'}`); }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/ballerina-extension/test/ai/evals/code/result-management/report-generator.ts` around lines 166 - 169, The console.log in the result handling for result.contextRetrievalEvaluation prints both an emoji and the boolean value (using ctx.is_relevant); remove the redundant boolean display by changing the log to only show the emoji (or emoji plus descriptive label) — locate the block where result.contextRetrievalEvaluation is assigned to ctx and update the console.log that currently reads `Context Retrieval: ${ctx.is_relevant ? '✅' : '❌'} Relevant: ${ctx.is_relevant}` to a single concise output such as `Context Retrieval: ${ctx.is_relevant ? '✅' : '❌'}`.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/ballerina-extension/src/features/ai/agent/prompts.ts`:
- Around line 179-199: The system/user prompt mismatch: getSystemPrompt() states
the agent will receive either a "Codebase High Level Overview" or full source in
<codebase_structure>, but getUserPrompt() injects a "<Codebase High Level
Summary>" (codeMapMarkdown) and omits <codebase_structure>, which can make the
agent assume full source is present. Update the prompt construction so they are
consistent: modify getSystemPrompt() and/or getUserPrompt() to use the same
terminology and explicit contract (either always provide <codebase_structure>
when mentioning full source or change getSystemPrompt() to reference "<Codebase
High Level Summary>"/codeMapMarkdown), and ensure any logic that sets
codeMapMarkdown vs. codebase_structure toggles the corresponding text blocks;
search for and update the prompt assembly functions getSystemPrompt and
getUserPrompt (and related template builders) to keep wording and conditional
injection aligned.
In `@packages/ballerina-extension/src/features/ai/agent/tools/glob.ts`:
- Around line 95-102: The ripgrep subprocess result (proc from spawnSync in
glob.ts) must be checked for proc.error before treating null/empty stdout as "No
matches"; add a guard in the function that calls spawnSync/getRgExecutable to
detect proc.error (including buffer-limit or spawn failures) and return
fail(...) with the proc.error message/details instead of falling through to the
RgExitCode.NoMatches branch; apply the same proc.error check to the analogous
grep tool implementation so real tool failures aren't masked as "no files
found".
In `@packages/ballerina-extension/src/features/ai/agent/tools/grep.ts`:
- Around line 148-157: The current matchCount overcounts because matchLines uses
/^\d+[:\-]/ which treats context lines prefixed with "-" as matches; update the
logic that builds matchLines (used when constructing the GrepResult object in
this block) to only count true match lines by matching line numbers followed by
':' (e.g. /^\d+:/) so matchCount reflects actual matches; keep the
displayed/truncation behavior unchanged and ensure the GrepResult.matchCount
uses the new filtered array length.
In `@packages/ballerina-extension/src/features/ai/agent/tools/text-editor.ts`:
- Line 927: Fix the typos in the tool's user-facing description string in
packages/ballerina-extension/src/features/ai/agent/tools/text-editor.ts: change
"insted" to "instead" and "that components" to "those components". Locate the
description used when registering or defining the tool (search for the sentence
fragment "From the Codebase High Level Summary, If you know the line range of
components to read," or the function/constant that registers the tool, e.g., the
tool's description parameter) and update the string accordingly.
In `@packages/ballerina-extension/src/features/ai/agent/tools/utils/rg-utils.ts`:
- Around line 27-30: The current logic returns the full stdout from
spawnSync(whichCmd, ["rg"], ...) which on Windows ("where") can contain multiple
paths; change the return to parse result.stdout, split by newlines, trim each
line and return only the first non-empty path (use the existing whichCmd/result
variables and the same spawnSync call), so subsequent spawnSync invocations use
a single valid executable path (apply this normalization for both win32 and
non-win32 cases).
In `@packages/ballerina-extension/test/ai/evals/code/utils/evaluator-utils.ts`:
- Line 310: The code unsafely casts toolCall.input to
ContextRetrievalEvaluationResult; add a runtime validation step (e.g., use the
existing Zod schema or a custom type guard) to verify shape before casting: call
schema.safeParse(toolCall.input) (or run the guard) and handle the failure path
(log/throw) if validation fails, otherwise assign evaluationResult =
parsed.data; update any references to evaluationResult to rely on the validated
value (symbols: toolCall.input, ContextRetrievalEvaluationResult,
evaluationResult).
In `@packages/ballerina-extension/test/data/healthcare_sample/mapping.bal`:
- Line 45: The mapping uses direct first-element access on patient.name[0].given
in mapGivenToName which will panic if name or given arrays are empty; update the
mapping to guard before dereferencing by checking patient.name is not empty and
patient.name[0].given is not empty (or use safe navigation/optional access) and
only call mapGivenToName when those checks pass; apply the same pattern for the
other occurrences referenced around lines 53-56 (e.g., any uses of
patient.name[0] or patient.name[0].given).
- Line 34: The fallback literal for patient gender is misspelled and can break
the enum cast in the expression using <uscore311:USCorePatientProfileGender> and
patient?.gender; change the fallback from "unkown" to the correct "unknown" (or
to the exact enum member name expected by
<uscore311:USCorePatientProfileGender>) so the cast succeeds when patient.gender
is absent.
- Around line 27-32: The mapping currently generates id and identifier[0].value
independently (using patient?.id ?: uuid:createType1AsString() and patient?.id
?: generatePatientId()), causing mismatches when source id is absent; fix by
computing a single local variable (e.g., generatedId) once using the fallback
logic (patient?.id ?: ...) and use that variable for both the id field and
identifier[0].value so both values are consistent in one mapping pass.
In `@packages/ballerina-extension/test/data/healthcare_sample/service.bal`:
- Around line 131-133: The Location header currently builds
`${SERVER_BASE_URL}/Patient/${dbResponse[0]}` which is incorrect for this
service; update the response.addHeader call to return the full exposed resource
path by including the service prefix (e.g. use
`${SERVER_BASE_URL}/fhir/r4/Patient/${dbResponse[0]}`) so that callers are
directed to the actual route; modify the response.addHeader(http:LOCATION, ...)
invocation (and keep response.statusCode = http:STATUS_CREATED) to use the
corrected path.
In `@packages/ballerina-extension/test/data/hotel_reservation/utils.bal`:
- Around line 48-57: The overlap predicate in getAllocatedRooms is too
restrictive and misses partial overlaps; replace the where clause that uses
"userCheckinUTC <= rCheckin && userCheckoutUTC >= rCheckout" with the standard
interval-overlap check using the variables userCheckinUTC, userCheckoutUTC,
rCheckin and rCheckout (i.e., ensure the condition enforces userCheckinUTC <
rCheckout && userCheckoutUTC > rCheckin) so roomReservations that partially
overlap the requested window are also returned.
In
`@packages/ballerina-extension/test/data/order_management_system/order_service/Ballerina.toml`:
- Around line 7-10: Update the dependency org for the order_utils entry in the
order_service Ballerina.toml so it matches the workspace package and imports:
change the [[dependency]] block that currently has org = "yasithrashan" name =
"order_utils" to use org = "wso2" (or remove the dependency so the local
workspace package is used), ensuring it aligns with order_utils/Ballerina.toml
and the import statement in order_service/functions.bal (import
wso2/order_utils;).
In
`@packages/ballerina-extension/test/data/order_management_system/order_service/modules/db/db_operations.bal`:
- Around line 12-20: The call to resultStream.next() can early-return on error
and skip resultStream.close(), leaking the stream; update the code around the
dbClient->query(...) resultStream so you call resultStream.close() in all paths
(both success and error) — e.g. capture the result of resultStream.next() into a
var, check whether it returned an error and if so call resultStream.close()
before propagating the error, or use a try/finally-style pattern to ensure
resultStream.close() is always executed after using resultStream (referencing
resultStream.next() and resultStream.close()).
In
`@packages/ballerina-extension/test/data/order_management_system/order_utils/utils.bal`:
- Around line 23-24: The function calculateLineTotal currently allows negative
quantities; add a guard at the start of calculateLineTotal to validate that
quantity is >= 0 and fail fast if not (e.g., panic or abort with a clear message
that includes the invalid quantity), then proceed to compute and return
unitPrice * <decimal>quantity; reference the calculateLineTotal function and the
parameters unitPrice and quantity when applying this check.
In
`@packages/ballerina-extension/test/data/salesforce_slack_integration_errors/functions.bal`:
- Line 190: The call that builds the Slack lookup URL concatenates email
directly into slackHttpClient->get("/api/users.lookupByEmail?email=" + email) so
reserved characters may break the request; import the URL module (import
ballerina/url as url), call url:encode(email) and use check to capture errors
(e.g., string encodedEmail = check url:encode(email)), then pass the
encodedEmail into slackHttpClient->get instead of the raw email to ensure a
correct, encoded query string.
In
`@packages/ballerina-extension/test/data/salesforce_slack_integration_errors/main.bal`:
- Around line 67-74: The current onUpdate flow marks processedLeadIds[leadId] =
true inside the initial lock before downstream work, which causes failed
processing to be permanently skipped; change the logic so the lock only reserves
the lead (e.g., insert an "in-flight" marker or a temporary placeholder) to
prevent concurrent processing, release the lock, run the downstream queries and
Slack notification, and only upon successful completion set
processedLeadIds[leadId] = true (or replace the in-flight marker with the final
marker) inside another short lock; ensure that on any error you remove the
in-flight reservation (or do not commit the final mark) so retries can
succeed—refer to processedLeadIds, the lock block, and onUpdate to locate where
to implement this change.
In
`@packages/ballerina-extension/test/data/salesforce_slack_integration/functions.bal`:
- Around line 184-194: The Slack lookup URL concatenates the raw email in
getSlackUserIdFromEmail which can break if the email contains reserved
characters; before calling
slackHttpClient->get("/api/users.lookupByEmail?email=" + email) encode the email
using ballerina/url:encode (or the equivalent URL-encoding helper) and use the
encoded value in the query string so the request is safe for all valid email
characters.
In
`@packages/ballerina-extension/test/data/salesforce_slack_integration/main.bal`:
- Around line 67-74: The code records processedLeadIds[leadId] inside the lock
before downstream work succeeds (in the lock block that checks
processedLeadIds.hasKey), which causes permanent skips on transient failures and
unbounded growth; change the logic in onUpdate so you only mark
processedLeadIds[leadId] = true after all downstream actions (Salesforce
queries/Slack send) complete successfully (still using the lock to guard the
check-and-set), and if you must set it earlier ensure you remove the entry on
any failure inside the same lock; additionally implement a bounded eviction/TTL
mechanism for processedLeadIds (e.g., time-based expiry or size cap with
LRU/queue) to prevent unbounded memory growth.
---
Nitpick comments:
In
`@packages/ballerina-extension/test/ai/evals/code/result-management/report-generator.ts`:
- Around line 192-195: The console.log in report-generator.ts is printing both
an emoji and the boolean for context relevance (using ctx.is_relevant) which is
redundant; update the logging inside the block that checks
result.contextRetrievalEvaluation so it only prints the emoji/state (e.g., keep
`Context Retrieval: ${ctx.is_relevant ? '✅' : '❌'}`) and remove the trailing
"Relevant: ${ctx.is_relevant}" portion, modifying the statement that references
ctx and ctx.is_relevant accordingly.
- Around line 166-169: The console.log in the result handling for
result.contextRetrievalEvaluation prints both an emoji and the boolean value
(using ctx.is_relevant); remove the redundant boolean display by changing the
log to only show the emoji (or emoji plus descriptive label) — locate the block
where result.contextRetrievalEvaluation is assigned to ctx and update the
console.log that currently reads `Context Retrieval: ${ctx.is_relevant ? '✅' :
'❌'} Relevant: ${ctx.is_relevant}` to a single concise output such as `Context
Retrieval: ${ctx.is_relevant ? '✅' : '❌'}`.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 692ebaae-dd3f-41e1-8efa-e62b09be16a1
⛔ Files ignored due to path filters (1)
common/config/rush/pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (81)
packages/ballerina-core/src/interfaces/extended-lang-client.tspackages/ballerina-extension/package.jsonpackages/ballerina-extension/src/core/extended-language-client.tspackages/ballerina-extension/src/features/ai/activator.tspackages/ballerina-extension/src/features/ai/agent/AgentExecutor.tspackages/ballerina-extension/src/features/ai/agent/index.tspackages/ballerina-extension/src/features/ai/agent/prompts.tspackages/ballerina-extension/src/features/ai/agent/tool-registry.tspackages/ballerina-extension/src/features/ai/agent/tools/glob.tspackages/ballerina-extension/src/features/ai/agent/tools/grep.tspackages/ballerina-extension/src/features/ai/agent/tools/text-editor.tspackages/ballerina-extension/src/features/ai/agent/tools/utils/rg-utils.tspackages/ballerina-extension/src/features/ai/executors/base/AICommandExecutor.tspackages/ballerina-extension/src/features/ai/utils/project/temp-project.tspackages/ballerina-extension/test/ai/evals/code/result-management/report-generator.tspackages/ballerina-extension/test/ai/evals/code/result-management/result-conversion.tspackages/ballerina-extension/test/ai/evals/code/result-management/result-persistence.tspackages/ballerina-extension/test/ai/evals/code/test-cases.tspackages/ballerina-extension/test/ai/evals/code/types/result-types.tspackages/ballerina-extension/test/ai/evals/code/types/test-types.tspackages/ballerina-extension/test/ai/evals/code/utils/evaluator-utils.tspackages/ballerina-extension/test/ai/evals/code/utils/test-validation.tspackages/ballerina-extension/test/data/healthcare_sample/Ballerina.tomlpackages/ballerina-extension/test/data/healthcare_sample/Config.tomlpackages/ballerina-extension/test/data/healthcare_sample/encounter_api_config.balpackages/ballerina-extension/test/data/healthcare_sample/mapping.balpackages/ballerina-extension/test/data/healthcare_sample/modules/db/persist_client.balpackages/ballerina-extension/test/data/healthcare_sample/modules/db/persist_db_config.balpackages/ballerina-extension/test/data/healthcare_sample/modules/db/persist_types.balpackages/ballerina-extension/test/data/healthcare_sample/modules/db/script.sqlpackages/ballerina-extension/test/data/healthcare_sample/patient_api_config.balpackages/ballerina-extension/test/data/healthcare_sample/persist/model.balpackages/ballerina-extension/test/data/healthcare_sample/service.balpackages/ballerina-extension/test/data/hotel_reservation/Ballerina.tomlpackages/ballerina-extension/test/data/hotel_reservation/service.balpackages/ballerina-extension/test/data/hotel_reservation/tests/Config.tomlpackages/ballerina-extension/test/data/hotel_reservation/tests/service_test.balpackages/ballerina-extension/test/data/hotel_reservation/types.balpackages/ballerina-extension/test/data/hotel_reservation/utils.balpackages/ballerina-extension/test/data/order_management_system/Ballerina.tomlpackages/ballerina-extension/test/data/order_management_system/order_service/Ballerina.tomlpackages/ballerina-extension/test/data/order_management_system/order_service/configurations.balpackages/ballerina-extension/test/data/order_management_system/order_service/functions.balpackages/ballerina-extension/test/data/order_management_system/order_service/main.balpackages/ballerina-extension/test/data/order_management_system/order_service/modules/db/db_client.balpackages/ballerina-extension/test/data/order_management_system/order_service/modules/db/db_config.balpackages/ballerina-extension/test/data/order_management_system/order_service/modules/db/db_operations.balpackages/ballerina-extension/test/data/order_management_system/order_service/modules/db/db_types.balpackages/ballerina-extension/test/data/order_management_system/order_service/modules/messaging/kafka_config.balpackages/ballerina-extension/test/data/order_management_system/order_service/modules/messaging/kafka_operations.balpackages/ballerina-extension/test/data/order_management_system/order_service/modules/messaging/kafka_producer.balpackages/ballerina-extension/test/data/order_management_system/order_service/modules/messaging/kafka_types.balpackages/ballerina-extension/test/data/order_management_system/order_service/service.balpackages/ballerina-extension/test/data/order_management_system/order_service/types.balpackages/ballerina-extension/test/data/order_management_system/order_utils/Ballerina.tomlpackages/ballerina-extension/test/data/order_management_system/order_utils/utils.balpackages/ballerina-extension/test/data/salesforce_slack_integration/Ballerina.tomlpackages/ballerina-extension/test/data/salesforce_slack_integration/agents.balpackages/ballerina-extension/test/data/salesforce_slack_integration/config.balpackages/ballerina-extension/test/data/salesforce_slack_integration/connections.balpackages/ballerina-extension/test/data/salesforce_slack_integration/data_mappings.balpackages/ballerina-extension/test/data/salesforce_slack_integration/functions.balpackages/ballerina-extension/test/data/salesforce_slack_integration/main.balpackages/ballerina-extension/test/data/salesforce_slack_integration/types.balpackages/ballerina-extension/test/data/salesforce_slack_integration_errors/Ballerina.tomlpackages/ballerina-extension/test/data/salesforce_slack_integration_errors/agents.balpackages/ballerina-extension/test/data/salesforce_slack_integration_errors/config.balpackages/ballerina-extension/test/data/salesforce_slack_integration_errors/connections.balpackages/ballerina-extension/test/data/salesforce_slack_integration_errors/data_mappings.balpackages/ballerina-extension/test/data/salesforce_slack_integration_errors/functions.balpackages/ballerina-extension/test/data/salesforce_slack_integration_errors/main.balpackages/ballerina-extension/test/data/salesforce_slack_integration_errors/types.balpackages/ballerina-extension/test/data/shopify_stripe_integration_errors/Ballerina.tomlpackages/ballerina-extension/test/data/shopify_stripe_integration_errors/agents.balpackages/ballerina-extension/test/data/shopify_stripe_integration_errors/automation.balpackages/ballerina-extension/test/data/shopify_stripe_integration_errors/config.balpackages/ballerina-extension/test/data/shopify_stripe_integration_errors/connections.balpackages/ballerina-extension/test/data/shopify_stripe_integration_errors/data_mappings.balpackages/ballerina-extension/test/data/shopify_stripe_integration_errors/functions.balpackages/ballerina-extension/test/data/shopify_stripe_integration_errors/main.balpackages/ballerina-extension/test/data/shopify_stripe_integration_errors/types.bal
| # Codebase Exploration | ||
| - When the user submits a query, you will receive either **Codebase High Level Overview** or **Complete Structure of the Codebase**. Identify which one you have received before proceeding. | ||
| - If you received Codebase High Level Overview, use it as a navigation map to locate the relevant components to the user query, in the codebase, but the actual source must be read separately when needed. | ||
| - Codebase High Level Overview lists, for each Ballerina file, all of its components (imports, configurables, variables, types, functions, services, listeners, classes) with their signatures and line ranges, but excludes implementation bodies, test files, and resource files. | ||
| - If you receive complete structure of the codebase, it contains the complete source of all .bal files (test and resource files excluded) provided directly in your context. | ||
|
|
||
| ## Context Retrieval | ||
| - Explore the codebase with ${GREP_TOOL_NAME}, ${FILE_READ_TOOL_NAME}, and ${GLOB_TOOL_NAME}, and keep exploring until you have all the context required to answer confidently. | ||
|
|
||
| ### Rules for exploration | ||
| - **DO NOT** guess the implementation based on signatures from Codebase High Level Overview or excerpts you retrieved. Always read the actual source code before using any information about a component in the codebase. This is critical to avoid hallucinations and wrong assumptions. | ||
| - When you update or write code, Codebase High Level Overview will become outdated. | ||
|
|
||
| ## File Modifications and Component Modifications | ||
| - You must apply changes to the existing source code using the provided ${[ | ||
| FILE_BATCH_EDIT_TOOL_NAME, | ||
| FILE_SINGLE_EDIT_TOOL_NAME, | ||
| FILE_WRITE_TOOL_NAME, | ||
| ].join( | ||
| ", " | ||
| )} tools. The complete existing source code will be provided in the <existing_code> section of the user prompt. | ||
| FILE_BATCH_EDIT_TOOL_NAME, | ||
| FILE_SINGLE_EDIT_TOOL_NAME, | ||
| FILE_WRITE_TOOL_NAME, | ||
| ].join( | ||
| ", " | ||
| )} tools. The complete existing source code will be provided in the <codebase_structure> section of the user prompt. |
There was a problem hiding this comment.
Keep the prompt contract consistent with the context you actually inject.
getSystemPrompt() says the agent will receive either a “Codebase High Level Overview” or full source in <codebase_structure>, but getUserPrompt() injects <Codebase High Level Summary> and omits <codebase_structure> entirely when codeMapMarkdown is present. That contradiction can make the agent assume source is already in context and skip required reads.
Suggested direction
- - When you update or write code, Codebase High Level Overview will become outdated.
+ - When you update or write code, any high-level summary becomes outdated.
- - You must apply changes to the existing source code using the provided ... tools. The complete existing source code will be provided in the <codebase_structure> section of the user prompt.
+ - You will receive either <Codebase High Level Summary> or <codebase_structure>.
+ - If only <Codebase High Level Summary> is present, use it for navigation only and read the relevant files before relying on implementation details.
+ - You must apply changes to the existing source code using the provided ... tools.Also applies to: 257-271
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/ballerina-extension/src/features/ai/agent/prompts.ts` around lines
179 - 199, The system/user prompt mismatch: getSystemPrompt() states the agent
will receive either a "Codebase High Level Overview" or full source in
<codebase_structure>, but getUserPrompt() injects a "<Codebase High Level
Summary>" (codeMapMarkdown) and omits <codebase_structure>, which can make the
agent assume full source is present. Update the prompt construction so they are
consistent: modify getSystemPrompt() and/or getUserPrompt() to use the same
terminology and explicit contract (either always provide <codebase_structure>
when mentioning full source or change getSystemPrompt() to reference "<Codebase
High Level Summary>"/codeMapMarkdown), and ensure any logic that sets
codeMapMarkdown vs. codebase_structure toggles the corresponding text blocks;
search for and update the prompt assembly functions getSystemPrompt and
getUserPrompt (and related template builders) to keep wording and conditional
injection aligned.
| const proc = spawnSync(getRgExecutable(), args, { encoding: "utf-8", maxBuffer: 10 * 1024 * 1024 }); | ||
|
|
||
| if (proc.status === RgExitCode.Error) { | ||
| const errMsg = (proc.stderr || "").trim(); | ||
| return fail(`ripgrep error: ${errMsg}`, errMsg); | ||
| } | ||
|
|
||
| if (proc.status === RgExitCode.NoMatches || !proc.stdout || proc.stdout.trim().length === 0) { |
There was a problem hiding this comment.
Handle ripgrep process failures before returning a no-match result.
When spawnSync() cannot start the process or hits the buffer limit, it sets error and leaves status null. This branch currently turns that into a successful “No files found” response, which hides real tool failures. The same guard is needed in the grep tool as well.
Proposed fix
const proc = spawnSync(getRgExecutable(), args, { encoding: "utf-8", maxBuffer: 10 * 1024 * 1024 });
+ if (proc.error || proc.status == null) {
+ const errMsg = proc.error?.message || (proc.stderr || "").trim() || "Failed to execute ripgrep";
+ return fail(`ripgrep error: ${errMsg}`, errMsg);
+ }
+
if (proc.status === RgExitCode.Error) {
const errMsg = (proc.stderr || "").trim();
return fail(`ripgrep error: ${errMsg}`, errMsg);
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const proc = spawnSync(getRgExecutable(), args, { encoding: "utf-8", maxBuffer: 10 * 1024 * 1024 }); | |
| if (proc.status === RgExitCode.Error) { | |
| const errMsg = (proc.stderr || "").trim(); | |
| return fail(`ripgrep error: ${errMsg}`, errMsg); | |
| } | |
| if (proc.status === RgExitCode.NoMatches || !proc.stdout || proc.stdout.trim().length === 0) { | |
| const proc = spawnSync(getRgExecutable(), args, { encoding: "utf-8", maxBuffer: 10 * 1024 * 1024 }); | |
| if (proc.error || proc.status == null) { | |
| const errMsg = proc.error?.message || (proc.stderr || "").trim() || "Failed to execute ripgrep"; | |
| return fail(`ripgrep error: ${errMsg}`, errMsg); | |
| } | |
| if (proc.status === RgExitCode.Error) { | |
| const errMsg = (proc.stderr || "").trim(); | |
| return fail(`ripgrep error: ${errMsg}`, errMsg); | |
| } | |
| if (proc.status === RgExitCode.NoMatches || !proc.stdout || proc.stdout.trim().length === 0) { |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/ballerina-extension/src/features/ai/agent/tools/glob.ts` around
lines 95 - 102, The ripgrep subprocess result (proc from spawnSync in glob.ts)
must be checked for proc.error before treating null/empty stdout as "No
matches"; add a guard in the function that calls spawnSync/getRgExecutable to
detect proc.error (including buffer-limit or spawn failures) and return
fail(...) with the proc.error message/details instead of falling through to the
RgExitCode.NoMatches branch; apply the same proc.error check to the analogous
grep tool implementation so real tool failures aren't masked as "no files
found".
| const truncated = lines.length > MAX_LINES; | ||
| const displayed = truncated ? lines.slice(0, MAX_LINES) : lines; | ||
| const truncationNote = truncated ? `\n... (truncated, showing ${MAX_LINES} of ${lines.length} lines)` : ""; | ||
| const matchLines = lines.filter((l) => /^\d+[:\-]/.test(l)); | ||
|
|
||
| const result: GrepResult = { | ||
| success: true, | ||
| message: displayed.join("\n") + truncationNote, | ||
| pattern, | ||
| matchCount: matchLines.length, |
There was a problem hiding this comment.
Count only actual matches in matchCount.
With -C 2, ripgrep prefixes context lines with -, so /^\d+[:\-]/ overcounts whenever context is returned. Restrict this to : lines if callers need an exact match count.
Proposed fix
- const matchLines = lines.filter((l) => /^\d+[:\-]/.test(l));
+ const matchLines = lines.filter((l) => /^\d+:/.test(l));📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const truncated = lines.length > MAX_LINES; | |
| const displayed = truncated ? lines.slice(0, MAX_LINES) : lines; | |
| const truncationNote = truncated ? `\n... (truncated, showing ${MAX_LINES} of ${lines.length} lines)` : ""; | |
| const matchLines = lines.filter((l) => /^\d+[:\-]/.test(l)); | |
| const result: GrepResult = { | |
| success: true, | |
| message: displayed.join("\n") + truncationNote, | |
| pattern, | |
| matchCount: matchLines.length, | |
| const truncated = lines.length > MAX_LINES; | |
| const displayed = truncated ? lines.slice(0, MAX_LINES) : lines; | |
| const truncationNote = truncated ? `\n... (truncated, showing ${MAX_LINES} of ${lines.length} lines)` : ""; | |
| const matchLines = lines.filter((l) => /^\d+:/.test(l)); | |
| const result: GrepResult = { | |
| success: true, | |
| message: displayed.join("\n") + truncationNote, | |
| pattern, | |
| matchCount: matchLines.length, |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/ballerina-extension/src/features/ai/agent/tools/grep.ts` around
lines 148 - 157, The current matchCount overcounts because matchLines uses
/^\d+[:\-]/ which treats context lines prefixed with "-" as matches; update the
logic that builds matchLines (used when constructing the GrepResult object in
this block) to only count true match lines by matching line numbers followed by
':' (e.g. /^\d+:/) so matchCount reflects actual matches; keep the
displayed/truncation behavior unchanged and ensure the GrepResult.matchCount
uses the new filtered array length.
| - For workspace projects, include the package directory prefix in the file path (e.g., "myPackage/main.bal"). | ||
| - You can optionally specify a line offset and limit (especially handy for long files). | ||
| - Any lines longer than 2000 characters will be truncated | ||
| - From the Codebase High Level Summary, If you know the line range of components to read, **ALWAYS** use the startLine and endLine parameters to read that components insted of files. |
There was a problem hiding this comment.
Fix typos in tool description.
Two typos in the user-facing description:
- "insted" should be "instead"
- "that components" should be "those components"
✏️ Proposed fix
- - From the Codebase High Level Summary, If you know the line range of components to read, **ALWAYS** use the startLine and endLine parameters to read that components insted of files.
+ - From the Codebase High Level Summary, If you know the line range of components to read, **ALWAYS** use the startLine and endLine parameters to read those components instead of files.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| - From the Codebase High Level Summary, If you know the line range of components to read, **ALWAYS** use the startLine and endLine parameters to read that components insted of files. | |
| - From the Codebase High Level Summary, If you know the line range of components to read, **ALWAYS** use the startLine and endLine parameters to read those components instead of files. |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/ballerina-extension/src/features/ai/agent/tools/text-editor.ts` at
line 927, Fix the typos in the tool's user-facing description string in
packages/ballerina-extension/src/features/ai/agent/tools/text-editor.ts: change
"insted" to "instead" and "that components" to "those components". Locate the
description used when registering or defining the tool (search for the sentence
fragment "From the Codebase High Level Summary, If you know the line range of
components to read," or the function/constant that registers the tool, e.g., the
tool's description parameter) and update the string accordingly.
| const whichCmd = process.platform === "win32" ? "where" : "which"; | ||
| const result = spawnSync(whichCmd, ["rg"], { encoding: "utf-8" }); | ||
| if (result.status === 0 && result.stdout?.trim()) { | ||
| return result.stdout.trim(); |
There was a problem hiding this comment.
Return only the first discovered rg path.
where rg can emit multiple matches on Windows. Returning the full stdout makes later spawnSync() calls use an invalid executable path and breaks both search tools when more than one installation is present.
Proposed fix
- if (result.status === 0 && result.stdout?.trim()) {
- return result.stdout.trim();
+ const candidates = result.stdout
+ ?.split(/\r?\n/)
+ .map((line) => line.trim())
+ .filter(Boolean);
+ if (result.status === 0 && candidates && candidates.length > 0) {
+ return candidates[0];
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const whichCmd = process.platform === "win32" ? "where" : "which"; | |
| const result = spawnSync(whichCmd, ["rg"], { encoding: "utf-8" }); | |
| if (result.status === 0 && result.stdout?.trim()) { | |
| return result.stdout.trim(); | |
| const whichCmd = process.platform === "win32" ? "where" : "which"; | |
| const result = spawnSync(whichCmd, ["rg"], { encoding: "utf-8" }); | |
| const candidates = result.stdout | |
| ?.split(/\r?\n/) | |
| .map((line) => line.trim()) | |
| .filter(Boolean); | |
| if (result.status === 0 && candidates && candidates.length > 0) { | |
| return candidates[0]; | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/ballerina-extension/src/features/ai/agent/tools/utils/rg-utils.ts`
around lines 27 - 30, The current logic returns the full stdout from
spawnSync(whichCmd, ["rg"], ...) which on Windows ("where") can contain multiple
paths; change the return to parse result.stdout, split by newlines, trim each
line and return only the first non-empty path (use the existing whichCmd/result
variables and the same spawnSync call), so subsequent spawnSync invocations use
a single valid executable path (apply this normalization for both win32 and
non-win32 cases).
| public function calculateLineTotal(decimal unitPrice, int quantity) returns decimal { | ||
| return unitPrice * <decimal>quantity; |
There was a problem hiding this comment.
Validate quantity before computing line totals.
calculateLineTotal currently permits negative quantities, which can produce invalid negative line totals. Add a guard and fail fast for invalid input.
Proposed fix
public function calculateLineTotal(decimal unitPrice, int quantity) returns decimal {
+ if quantity < 0 {
+ panic error("Quantity must be non-negative");
+ }
return unitPrice * <decimal>quantity;
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| public function calculateLineTotal(decimal unitPrice, int quantity) returns decimal { | |
| return unitPrice * <decimal>quantity; | |
| public function calculateLineTotal(decimal unitPrice, int quantity) returns decimal { | |
| if quantity < 0 { | |
| panic error("Quantity must be non-negative"); | |
| } | |
| return unitPrice * <decimal>quantity; | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@packages/ballerina-extension/test/data/order_management_system/order_utils/utils.bal`
around lines 23 - 24, The function calculateLineTotal currently allows negative
quantities; add a guard at the start of calculateLineTotal to validate that
quantity is >= 0 and fail fast if not (e.g., panic or abort with a clear message
that includes the invalid quantity), then proceed to compute and return
unitPrice * <decimal>quantity; reference the calculateLineTotal function and the
parameters unitPrice and quantity when applying this check.
| } | ||
|
|
||
| // Use raw HTTP client — slack connector incorrectly types the user field as an array | ||
| http:Response|error httpResponse = slackHttpClient->get("/api/users.lookupByEmail?email=" + email); |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
Ballerina ballerina/url module encode function signature
💡 Result:
Ballerina’s url module exposes encode/decode with this function signature: public function encode(string str, string charset) returns string|Error (And the corresponding decode is: public function decode(string str, string charset) returns string|Error.) [1] The module documentation describes url encoding/decoding behavior and that the API is provided by importing ballerina/url. [2][3]
Citations:
- 1: Update and refactor ballerina/encoding module APIs ballerina-platform/ballerina-library#907
- 2: https://central.ballerina.io/ballerina/url/2.6.1
- 3: https://ballerina.io/learn/by-example/url-encode-decode/
URL-encode email before appending it to the lookup query string (line 190).
email is concatenated directly into /api/users.lookupByEmail?email=... without URL encoding, so reserved characters can change how the endpoint interprets the parameter. Use url:encode (returns string|Error) and check.
🔧 Proposed fix
- http:Response|error httpResponse = slackHttpClient->get("/api/users.lookupByEmail?email=" + email);
+ string encodedEmail = check url:encode(email, "UTF-8");
+ http:Response|error httpResponse = slackHttpClient->get("/api/users.lookupByEmail?email=" + encodedEmail);Add the import:
+import ballerina/url;🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@packages/ballerina-extension/test/data/salesforce_slack_integration_errors/functions.bal`
at line 190, The call that builds the Slack lookup URL concatenates email
directly into slackHttpClient->get("/api/users.lookupByEmail?email=" + email) so
reserved characters may break the request; import the URL module (import
ballerina/url as url), call url:encode(email) and use check to capture errors
(e.g., string encodedEmail = check url:encode(email)), then pass the
encodedEmail into slackHttpClient->get instead of the raw email to ensure a
correct, encoded query string.
| // Deduplicate: skip if this lead was already processed (guards against duplicate events) | ||
| lock { | ||
| if processedLeadIds.hasKey(leadId) { | ||
| log:printInfo("Duplicate lead conversion event, skipping", leadId = leadId); | ||
| return; | ||
| } | ||
| processedLeadIds[leadId] = true; | ||
| } |
There was a problem hiding this comment.
Dedup marks the lead as processed before processing completes.
processedLeadIds[leadId] = true is set before the downstream queries and Slack notification run (Lines 79-154). If any of those steps return an error, onUpdate logs the failure but the lead is already recorded as processed, so a redelivered event will be skipped and the notification is lost. Consider marking the lead processed only after successful completion (while still guarding against concurrent in-flight duplicates).
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@packages/ballerina-extension/test/data/salesforce_slack_integration_errors/main.bal`
around lines 67 - 74, The current onUpdate flow marks processedLeadIds[leadId] =
true inside the initial lock before downstream work, which causes failed
processing to be permanently skipped; change the logic so the lock only reserves
the lead (e.g., insert an "in-flight" marker or a temporary placeholder) to
prevent concurrent processing, release the lock, run the downstream queries and
Slack notification, and only upon successful completion set
processedLeadIds[leadId] = true (or replace the in-flight marker with the final
marker) inside another short lock; ensure that on any error you remove the
in-flight reservation (or do not commit the final mark) so retries can
succeed—refer to processedLeadIds, the lock block, and onUpdate to locate where
to implement this change.
| function getSlackUserIdFromEmail(string? email) returns string?|error { | ||
| if email is () { | ||
| return (); | ||
| } | ||
|
|
||
| // Use raw HTTP client — slack connector incorrectly types the user field as an array | ||
| http:Response|error httpResponse = slackHttpClient->get("/api/users.lookupByEmail?email=" + email); | ||
| if httpResponse is error { | ||
| log:printWarn("Failed to lookup Slack user by email", 'error = httpResponse, email = email); | ||
| return (); | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
Ballerina ballerina/url module url:encode function signature
💡 Result:
Ballerina (ballerina/url) module: url:encode function signature is: public function encode(string str, string charset) returns string|Error {} This is the API kept in the ballerina/url module (renamed from ballerina/encoding) and is documented/defined as taking the URL string and a charset, and returning either a string or Error. [1] Notes: - The ballerina/url package is imported as: import ballerina/url; [2] - The module provides URL encoding/decoding functions. [3][2]
Citations:
- 1: Update and refactor ballerina/encoding module APIs ballerina-platform/ballerina-library#907
- 2: https://central.ballerina.io/ballerina/url/2.6.1
- 3: https://central.ballerina.io/ballerina/url/latest
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE="packages/ballerina-extension/test/data/salesforce_slack_integration/functions.bal"
# Show imports near top
rg -n "import ballerina/url;" "$FILE" || true
sed -n '1,80p' "$FILE"
# Show the function and surrounding lines
rg -n "function getSlackUserIdFromEmail" "$FILE"
sed -n '150,230p' "$FILE"Repository: wso2/ballerina-vscode
Length of output: 4949
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE="packages/ballerina-extension/test/data/salesforce_slack_integration/functions.bal"
echo "=== Check existing url imports ==="
rg -n "import ballerina/url;" "$FILE" || true
echo "=== Check existing url:encode usage ==="
rg -n "url:encode" "$FILE" || trueRepository: wso2/ballerina-vscode
Length of output: 140
Encode email before building the Slack lookup query URL
email is concatenated directly into .../users.lookupByEmail?email=...; reserved characters (e.g. +) will be interpreted as part of the query and can break valid lookups. Encode the parameter with ballerina/url:encode.
🔧 Suggested change
import ballerina/http;
import ballerina/lang.regexp;
import ballerina/log;
import ballerina/time;
+import ballerina/url;
// Get Slack user ID from email
function getSlackUserIdFromEmail(string? email) returns string?|error {
if email is () {
return ();
}
+ string|error encodedEmail = url:encode(email, "UTF-8");
+ if encodedEmail is error {
+ log:printWarn("Failed to encode Slack lookup email", 'error = encodedEmail, email = email);
+ return ();
+ }
+
// Use raw HTTP client — slack connector incorrectly types the user field as an array
- http:Response|error httpResponse = slackHttpClient->get("/api/users.lookupByEmail?email=" + email);
+ http:Response|error httpResponse = slackHttpClient->get("/api/users.lookupByEmail?email=" + encodedEmail);
if httpResponse is error {
log:printWarn("Failed to lookup Slack user by email", 'error = httpResponse, email = email);
return ();
}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@packages/ballerina-extension/test/data/salesforce_slack_integration/functions.bal`
around lines 184 - 194, The Slack lookup URL concatenates the raw email in
getSlackUserIdFromEmail which can break if the email contains reserved
characters; before calling
slackHttpClient->get("/api/users.lookupByEmail?email=" + email) encode the email
using ballerina/url:encode (or the equivalent URL-encoding helper) and use the
encoded value in the query string so the request is safe for all valid email
characters.
| // Deduplicate: skip if this lead was already processed (guards against duplicate events) | ||
| lock { | ||
| if processedLeadIds.hasKey(leadId) { | ||
| log:printInfo("Duplicate lead conversion event, skipping", leadId = leadId); | ||
| return; | ||
| } | ||
| processedLeadIds[leadId] = true; | ||
| } |
There was a problem hiding this comment.
Dedup entry is recorded before processing succeeds.
leadId is marked processed before the downstream queries and Slack send complete. If a later step fails, onUpdate logs the error but the entry remains, so a redelivered event will be skipped and never processed. Consider recording success only after completion (or removing the entry on failure), keeping the lock semantics in mind for concurrent duplicates.
Separately, processedLeadIds is never evicted, so it grows unbounded for a long-running listener. A bounded/TTL strategy would avoid steady memory growth.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/ballerina-extension/test/data/salesforce_slack_integration/main.bal`
around lines 67 - 74, The code records processedLeadIds[leadId] inside the lock
before downstream work succeeds (in the lock block that checks
processedLeadIds.hasKey), which causes permanent skips on transient failures and
unbounded growth; change the logic in onUpdate so you only mark
processedLeadIds[leadId] = true after all downstream actions (Salesforce
queries/Slack send) complete successfully (still using the lock to guard the
check-and-set), and if you must set it earlier ensure you remove the entry on
any failure inside the same lock; additionally implement a bounded eviction/TTL
mechanism for processedLeadIds (e.g., time-based expiry or size cap with
LRU/queue) to prevent unbounded memory growth.
Purpose
$title
Resolves: https://github.com/wso2-enterprise/integration-engineering/issues/893
key Changes
• Introduced CodeMap, a Markdown-based file that provides a high-level overview of the symbols in the Ballerina codebase.